Extracting and Visualizing Stock Data
Description
Extracting essential data from a dataset and displaying it is a necessary part of data science; therefore individuals can make correct decisions based on the data. In this assignment, you will extract some stock data, you will then display this data in a graph.
Table of Contents
- Define a Function that Makes a Graph
- Question 1: Use yfinance to Extract Stock Data
- Question 2: Use Webscraping to Extract Tesla Revenue Data
- Question 3: Use yfinance to Extract Stock Data
- Question 4: Use Webscraping to Extract GME Revenue Data
- Question 5: Plot Tesla Stock Graph
- Question 6: Plot GameStop Stock Graph
Estimated Time Needed: 30 min
Note:- If you are working Locally using anaconda, please uncomment the following code and execute it. Use the version as per your python version.
!pip install yfinance
!pip install bs4
!pip install nbformat
!pip install matplotlib
Collecting yfinance Downloading yfinance-1.2.0-py2.py3-none-any.whl.metadata (6.1 kB) Collecting pandas>=1.3.0 (from yfinance) Downloading pandas-3.0.1-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (79 kB) Collecting numpy>=1.16.5 (from yfinance) Downloading numpy-2.4.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (6.6 kB) Requirement already satisfied: requests>=2.31 in /opt/conda/lib/python3.12/site-packages (from yfinance) (2.32.3) Collecting multitasking>=0.0.7 (from yfinance) Downloading multitasking-0.0.12.tar.gz (19 kB) Preparing metadata (setup.py) ... done Requirement already satisfied: platformdirs>=2.0.0 in /opt/conda/lib/python3.12/site-packages (from yfinance) (4.3.6) Requirement already satisfied: pytz>=2022.5 in /opt/conda/lib/python3.12/site-packages (from yfinance) (2024.2) Requirement already satisfied: frozendict>=2.3.4 in /opt/conda/lib/python3.12/site-packages (from yfinance) (2.4.6) Collecting peewee>=3.16.2 (from yfinance) Downloading peewee-3.19.0-py3-none-any.whl.metadata (7.0 kB) Requirement already satisfied: beautifulsoup4>=4.11.1 in /opt/conda/lib/python3.12/site-packages (from yfinance) (4.12.3) Collecting curl_cffi<0.14,>=0.7 (from yfinance) Downloading curl_cffi-0.13.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (13 kB) Collecting protobuf>=3.19.0 (from yfinance) Downloading protobuf-6.33.5-cp39-abi3-manylinux2014_x86_64.whl.metadata (593 bytes) Collecting websockets>=13.0 (from yfinance) Downloading websockets-16.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.metadata (6.8 kB) Requirement already satisfied: soupsieve>1.2 in /opt/conda/lib/python3.12/site-packages (from beautifulsoup4>=4.11.1->yfinance) (2.5) Requirement already satisfied: cffi>=1.12.0 in /opt/conda/lib/python3.12/site-packages (from curl_cffi<0.14,>=0.7->yfinance) (1.17.1) Requirement already satisfied: certifi>=2024.2.2 in /opt/conda/lib/python3.12/site-packages (from curl_cffi<0.14,>=0.7->yfinance) (2024.12.14) Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.12/site-packages (from pandas>=1.3.0->yfinance) (2.9.0.post0) Requirement already satisfied: charset_normalizer<4,>=2 in /opt/conda/lib/python3.12/site-packages (from requests>=2.31->yfinance) (3.4.1) Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.12/site-packages (from requests>=2.31->yfinance) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.12/site-packages (from requests>=2.31->yfinance) (2.3.0) Requirement already satisfied: pycparser in /opt/conda/lib/python3.12/site-packages (from cffi>=1.12.0->curl_cffi<0.14,>=0.7->yfinance) (2.22) Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.12/site-packages (from python-dateutil>=2.8.2->pandas>=1.3.0->yfinance) (1.17.0) Downloading yfinance-1.2.0-py2.py3-none-any.whl (130 kB) Downloading curl_cffi-0.13.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.3/8.3 MB 131.9 MB/s eta 0:00:00 Downloading numpy-2.4.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 16.6/16.6 MB 203.6 MB/s eta 0:00:00 Downloading pandas-3.0.1-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (10.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.9/10.9 MB 187.1 MB/s eta 0:00:00 Downloading peewee-3.19.0-py3-none-any.whl (411 kB) Downloading protobuf-6.33.5-cp39-abi3-manylinux2014_x86_64.whl (323 kB) Downloading websockets-16.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (184 kB) Building wheels for collected packages: multitasking Building wheel for multitasking (setup.py) ... done Created wheel for multitasking: filename=multitasking-0.0.12-py3-none-any.whl size=15605 sha256=e1fadc20eba215b73a018b24184a32dff4d897602ccd107d0e1c996ab54d9eb7 Stored in directory: /home/jupyterlab/.cache/pip/wheels/cc/bd/6f/664d62c99327abeef7d86489e6631cbf45b56fbf7ef1d6ef00 Successfully built multitasking Installing collected packages: peewee, multitasking, websockets, protobuf, numpy, pandas, curl_cffi, yfinance Successfully installed curl_cffi-0.13.0 multitasking-0.0.12 numpy-2.4.2 pandas-3.0.1 peewee-3.19.0 protobuf-6.33.5 websockets-16.0 yfinance-1.2.0 Collecting bs4 Downloading bs4-0.0.2-py2.py3-none-any.whl.metadata (411 bytes) Requirement already satisfied: beautifulsoup4 in /opt/conda/lib/python3.12/site-packages (from bs4) (4.12.3) Requirement already satisfied: soupsieve>1.2 in /opt/conda/lib/python3.12/site-packages (from beautifulsoup4->bs4) (2.5) Downloading bs4-0.0.2-py2.py3-none-any.whl (1.2 kB) Installing collected packages: bs4 Successfully installed bs4-0.0.2 Requirement already satisfied: nbformat in /opt/conda/lib/python3.12/site-packages (5.10.4) Requirement already satisfied: fastjsonschema>=2.15 in /opt/conda/lib/python3.12/site-packages (from nbformat) (2.21.1) Requirement already satisfied: jsonschema>=2.6 in /opt/conda/lib/python3.12/site-packages (from nbformat) (4.23.0) Requirement already satisfied: jupyter-core!=5.0.*,>=4.12 in /opt/conda/lib/python3.12/site-packages (from nbformat) (5.7.2) Requirement already satisfied: traitlets>=5.1 in /opt/conda/lib/python3.12/site-packages (from nbformat) (5.14.3) Requirement already satisfied: attrs>=22.2.0 in /opt/conda/lib/python3.12/site-packages (from jsonschema>=2.6->nbformat) (25.1.0) Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /opt/conda/lib/python3.12/site-packages (from jsonschema>=2.6->nbformat) (2024.10.1) Requirement already satisfied: referencing>=0.28.4 in /opt/conda/lib/python3.12/site-packages (from jsonschema>=2.6->nbformat) (0.36.2) Requirement already satisfied: rpds-py>=0.7.1 in /opt/conda/lib/python3.12/site-packages (from jsonschema>=2.6->nbformat) (0.22.3) Requirement already satisfied: platformdirs>=2.5 in /opt/conda/lib/python3.12/site-packages (from jupyter-core!=5.0.*,>=4.12->nbformat) (4.3.6) Requirement already satisfied: typing-extensions>=4.4.0 in /opt/conda/lib/python3.12/site-packages (from referencing>=0.28.4->jsonschema>=2.6->nbformat) (4.12.2) Collecting matplotlib Downloading matplotlib-3.10.8-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (52 kB) Collecting contourpy>=1.0.1 (from matplotlib) Downloading contourpy-1.3.3-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (5.5 kB) Collecting cycler>=0.10 (from matplotlib) Downloading cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB) Collecting fonttools>=4.22.0 (from matplotlib) Downloading fonttools-4.61.1-cp312-cp312-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl.metadata (114 kB) Collecting kiwisolver>=1.3.1 (from matplotlib) Downloading kiwisolver-1.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (6.3 kB) Requirement already satisfied: numpy>=1.23 in /opt/conda/lib/python3.12/site-packages (from matplotlib) (2.4.2) Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.12/site-packages (from matplotlib) (24.2) Collecting pillow>=8 (from matplotlib) Downloading pillow-12.1.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (8.8 kB) Collecting pyparsing>=3 (from matplotlib) Downloading pyparsing-3.3.2-py3-none-any.whl.metadata (5.8 kB) Requirement already satisfied: python-dateutil>=2.7 in /opt/conda/lib/python3.12/site-packages (from matplotlib) (2.9.0.post0) Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.12/site-packages (from python-dateutil>=2.7->matplotlib) (1.17.0) Downloading matplotlib-3.10.8-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (8.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.7/8.7 MB 146.0 MB/s eta 0:00:00 Downloading contourpy-1.3.3-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (362 kB) Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB) Downloading fonttools-4.61.1-cp312-cp312-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl (5.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.0/5.0 MB 146.8 MB/s eta 0:00:00 Downloading kiwisolver-1.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.5/1.5 MB 95.7 MB/s eta 0:00:00 Downloading pillow-12.1.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (7.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.0/7.0 MB 181.4 MB/s eta 0:00:00 Downloading pyparsing-3.3.2-py3-none-any.whl (122 kB) Installing collected packages: pyparsing, pillow, kiwisolver, fonttools, cycler, contourpy, matplotlib Successfully installed contourpy-1.3.3 cycler-0.12.1 fonttools-4.61.1 kiwisolver-1.4.9 matplotlib-3.10.8 pillow-12.1.1 pyparsing-3.3.2
!pip install yfinance
!pip install plotly
!pip install requests
!pip imstall beautifulsoup4
!pip install html5lib
Requirement already satisfied: yfinance in /opt/conda/lib/python3.12/site-packages (1.2.0) Requirement already satisfied: pandas>=1.3.0 in /opt/conda/lib/python3.12/site-packages (from yfinance) (3.0.1) Requirement already satisfied: numpy>=1.16.5 in /opt/conda/lib/python3.12/site-packages (from yfinance) (2.4.2) Requirement already satisfied: requests>=2.31 in /opt/conda/lib/python3.12/site-packages (from yfinance) (2.32.3) Requirement already satisfied: multitasking>=0.0.7 in /opt/conda/lib/python3.12/site-packages (from yfinance) (0.0.12) Requirement already satisfied: platformdirs>=2.0.0 in /opt/conda/lib/python3.12/site-packages (from yfinance) (4.3.6) Requirement already satisfied: pytz>=2022.5 in /opt/conda/lib/python3.12/site-packages (from yfinance) (2024.2) Requirement already satisfied: frozendict>=2.3.4 in /opt/conda/lib/python3.12/site-packages (from yfinance) (2.4.6) Requirement already satisfied: peewee>=3.16.2 in /opt/conda/lib/python3.12/site-packages (from yfinance) (3.19.0) Requirement already satisfied: beautifulsoup4>=4.11.1 in /opt/conda/lib/python3.12/site-packages (from yfinance) (4.12.3) Requirement already satisfied: curl_cffi<0.14,>=0.7 in /opt/conda/lib/python3.12/site-packages (from yfinance) (0.13.0) Requirement already satisfied: protobuf>=3.19.0 in /opt/conda/lib/python3.12/site-packages (from yfinance) (6.33.5) Requirement already satisfied: websockets>=13.0 in /opt/conda/lib/python3.12/site-packages (from yfinance) (16.0) Requirement already satisfied: soupsieve>1.2 in /opt/conda/lib/python3.12/site-packages (from beautifulsoup4>=4.11.1->yfinance) (2.5) Requirement already satisfied: cffi>=1.12.0 in /opt/conda/lib/python3.12/site-packages (from curl_cffi<0.14,>=0.7->yfinance) (1.17.1) Requirement already satisfied: certifi>=2024.2.2 in /opt/conda/lib/python3.12/site-packages (from curl_cffi<0.14,>=0.7->yfinance) (2024.12.14) Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.12/site-packages (from pandas>=1.3.0->yfinance) (2.9.0.post0) Requirement already satisfied: charset_normalizer<4,>=2 in /opt/conda/lib/python3.12/site-packages (from requests>=2.31->yfinance) (3.4.1) Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.12/site-packages (from requests>=2.31->yfinance) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.12/site-packages (from requests>=2.31->yfinance) (2.3.0) Requirement already satisfied: pycparser in /opt/conda/lib/python3.12/site-packages (from cffi>=1.12.0->curl_cffi<0.14,>=0.7->yfinance) (2.22) Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.12/site-packages (from python-dateutil>=2.8.2->pandas>=1.3.0->yfinance) (1.17.0) Requirement already satisfied: plotly in /opt/conda/lib/python3.12/site-packages (5.24.1) Requirement already satisfied: tenacity>=6.2.0 in /opt/conda/lib/python3.12/site-packages (from plotly) (9.1.2) Requirement already satisfied: packaging in /opt/conda/lib/python3.12/site-packages (from plotly) (24.2) Requirement already satisfied: requests in /opt/conda/lib/python3.12/site-packages (2.32.3) Requirement already satisfied: charset_normalizer<4,>=2 in /opt/conda/lib/python3.12/site-packages (from requests) (3.4.1) Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.12/site-packages (from requests) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.12/site-packages (from requests) (2.3.0) Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.12/site-packages (from requests) (2024.12.14) ERROR: unknown command "imstall" - maybe you meant "install" Collecting html5lib Downloading html5lib-1.1-py2.py3-none-any.whl.metadata (16 kB) Requirement already satisfied: six>=1.9 in /opt/conda/lib/python3.12/site-packages (from html5lib) (1.17.0) Requirement already satisfied: webencodings in /opt/conda/lib/python3.12/site-packages (from html5lib) (0.5.1) Downloading html5lib-1.1-py2.py3-none-any.whl (112 kB) Installing collected packages: html5lib Successfully installed html5lib-1.1
import yfinance as yf
import pandas as pd
import requests
from bs4 import BeautifulSoup
In Python, you can ignore warnings using the warnings module. You can use the filterwarnings function to filter or ignore specific warning messages or categories.
import warnings
# Ignore all warnings
warnings.filterwarnings("ignore", category=FutureWarning)
Define Graphing Function¶
In this section, we define the function make_graph. You don't have to know how the function works, you should only care about the inputs. It takes a dataframe with stock data (dataframe must contain Date and Close columns), a dataframe with revenue data (dataframe must contain Date and Revenue columns), and the name of the stock.
# The make_graph function has been modified to use Matplotlib for static graphs. Earlier, it used Plotly to generate interactive dashboards, which caused issues when uploading the notebook in the MARK assignment submission.
import matplotlib.pyplot as plt
def make_graph(stock_data, revenue_data, stock):
stock_data_specific = stock_data[stock_data.Date <= '2021-06-14']
revenue_data_specific = revenue_data[revenue_data.Date <= '2021-04-30']
fig, axes = plt.subplots(2, 1, figsize=(12, 8), sharex=True)
# Stock price
axes[0].plot(pd.to_datetime(stock_data_specific.Date), stock_data_specific.Close.astype("float"), label="Share Price", color="blue")
axes[0].set_ylabel("Price ($US)")
axes[0].set_title(f"{stock} - Historical Share Price")
# Revenue
axes[1].plot(pd.to_datetime(revenue_data_specific.Date), revenue_data_specific.Revenue.astype("float"), label="Revenue", color="green")
axes[1].set_ylabel("Revenue ($US Millions)")
axes[1].set_xlabel("Date")
axes[1].set_title(f"{stock} - Historical Revenue")
plt.tight_layout()
plt.show()
!pip install matplotlib
Requirement already satisfied: matplotlib in /opt/conda/lib/python3.12/site-packages (3.10.8) Requirement already satisfied: contourpy>=1.0.1 in /opt/conda/lib/python3.12/site-packages (from matplotlib) (1.3.3) Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.12/site-packages (from matplotlib) (0.12.1) Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.12/site-packages (from matplotlib) (4.61.1) Requirement already satisfied: kiwisolver>=1.3.1 in /opt/conda/lib/python3.12/site-packages (from matplotlib) (1.4.9) Requirement already satisfied: numpy>=1.23 in /opt/conda/lib/python3.12/site-packages (from matplotlib) (2.4.2) Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.12/site-packages (from matplotlib) (24.2) Requirement already satisfied: pillow>=8 in /opt/conda/lib/python3.12/site-packages (from matplotlib) (12.1.1) Requirement already satisfied: pyparsing>=3 in /opt/conda/lib/python3.12/site-packages (from matplotlib) (3.3.2) Requirement already satisfied: python-dateutil>=2.7 in /opt/conda/lib/python3.12/site-packages (from matplotlib) (2.9.0.post0) Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.12/site-packages (from python-dateutil>=2.7->matplotlib) (1.17.0)
import matplotlib.pyplot as plt
Use the make_graph function that we’ve already defined. You’ll need to invoke it in questions 5 and 6 to display the graphs and create the dashboard.
Note: You don’t need to redefine the function for plotting graphs anywhere else in this notebook; just use the existing function.
Question 1: Use yfinance to Extract Stock Data¶
Using the Ticker function enter the ticker symbol of the stock we want to extract data on to create a ticker object. The stock is Tesla and its ticker symbol is TSLA.
import yfinance as yf
# Create a Ticker object for Tesla
tesla = yf.Ticker("TSLA")
Using the ticker object and the function history extract stock information and save it in a dataframe named tesla_data. Set the period parameter to "max" so we get information for the maximum amount of time.
import yfinance as yf
import pandas as pd
# Create Ticker object for Tesla
tesla = yf.Ticker("TSLA")
# Extract historical stock data for maximum period
tesla_data = tesla.history(period="max")
# Optional: reset the index to make 'Date' a column
tesla_data = tesla_data.reset_index()
# Display the first few rows
print(tesla_data.head())
Date Open High Low Close \
0 2010-06-29 00:00:00-04:00 1.266667 1.666667 1.169333 1.592667
1 2010-06-30 00:00:00-04:00 1.719333 2.028000 1.553333 1.588667
2 2010-07-01 00:00:00-04:00 1.666667 1.728000 1.351333 1.464000
3 2010-07-02 00:00:00-04:00 1.533333 1.540000 1.247333 1.280000
4 2010-07-06 00:00:00-04:00 1.333333 1.333333 1.055333 1.074000
Volume Dividends Stock Splits
0 281494500 0.0 0.0
1 257806500 0.0 0.0
2 123282000 0.0 0.0
3 77097000 0.0 0.0
4 103003500 0.0 0.0
Reset the index using the reset_index(inplace=True) function on the tesla_data DataFrame and display the first five rows of the tesla_data dataframe using the head function. Take a screenshot of the results and code from the beginning of Question 1 to the results below.
# Reset the index in-place so 'Date' becomes a column
tesla_data.reset_index(inplace=True)
# Display the first five rows
print(tesla_data.head())
index Date Open High Low Close \
0 0 2010-06-29 00:00:00-04:00 1.266667 1.666667 1.169333 1.592667
1 1 2010-06-30 00:00:00-04:00 1.719333 2.028000 1.553333 1.588667
2 2 2010-07-01 00:00:00-04:00 1.666667 1.728000 1.351333 1.464000
3 3 2010-07-02 00:00:00-04:00 1.533333 1.540000 1.247333 1.280000
4 4 2010-07-06 00:00:00-04:00 1.333333 1.333333 1.055333 1.074000
Volume Dividends Stock Splits
0 281494500 0.0 0.0
1 257806500 0.0 0.0
2 123282000 0.0 0.0
3 77097000 0.0 0.0
4 103003500 0.0 0.0
Question 2: Use Webscraping to Extract Tesla Revenue Data¶
import requests
# URL of the webpage
url = "https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/revenue.htm"
# Send GET request to download the webpage
response = requests.get(url)
# Save the HTML text of the response
html_data = response.text
# Optional: check the first 500 characters
print(html_data[:500])
<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]-->
<!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]-->
<!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<link rel="canonical" href="https://www.macrotrends.net/stocks/charts/TSLA/tesla/revenue" />
Parse the html data using beautiful_soup using parser i.e html5lib or html.parser.
import requests
from bs4 import BeautifulSoup
url = "https://example.com"
response = requests.get(url)
html_data = response.text # HTML stored here
soup = BeautifulSoup(html_data, "html.parser") # or "html5lib"
print(soup.prettify()[:500])
<!DOCTYPE html>
<html lang="en">
<head>
<title>
Example Domain
</title>
<meta content="width=device-width, initial-scale=1" name="viewport"/>
<style>
body{background:#eee;width:60vw;margin:15vh auto;font-family:system-ui,sans-serif}h1{font-size:1.5em}div{opacity:0.8}a:link,a:visited{color:#348}
</style>
</head>
<body>
<div>
<h1>
Example Domain
</h1>
<p>
This domain is for use in documentation examples without needing permission. Avoid use in operations.
<
import requests
url = "https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/revenue.htm"
response = requests.get(url)
html_data = response.text
!pip install html5lib
Requirement already satisfied: html5lib in /opt/conda/lib/python3.12/site-packages (1.1) Requirement already satisfied: six>=1.9 in /opt/conda/lib/python3.12/site-packages (from html5lib) (1.17.0) Requirement already satisfied: webencodings in /opt/conda/lib/python3.12/site-packages (from html5lib) (0.5.1)
Using BeautifulSoup or the read_html function extract the table with Tesla Revenue and store it into a dataframe named tesla_revenue. The dataframe should have columns Date and Revenue.
Step-by-step instructions
Here are the step-by-step instructions:
1. Create an Empty DataFrame
2. Find the Relevant Table
3. Check for the Tesla Quarterly Revenue Table
4. Iterate Through Rows in the Table Body
5. Extract Data from Columns
6. Append Data to the DataFrame
import requests
from bs4 import BeautifulSoup
import pandas as pd
# Request webpage
html_data = requests.get(url).text
# Parse HTML
soup = BeautifulSoup(html_data, "html.parser")
# Find table
table = soup.find("table")
# Extract rows
rows = table.find_all("tr")
data = []
for row in rows[1:]: # skip header
cols = row.find_all("td")
if len(cols) > 0:
date = cols[0].text.strip()
revenue = cols[1].text
data.append([data, revenue])
# Create DataFrame
tesla_revenue = pd.DataFrame(data, columns=["Date", "Revenue"])
# Clean Revenue column
tesla_revenue["Revenue"] = tesla_revenue["Revenue"].str.replace(",", "").str.replace("$", "")
tesla_revenue.head()
| Date | Revenue | |
|---|---|---|
| 0 | [[[[[[...], [[...], '$31,536'], [[...], '$24,5... | 53823 |
| 1 | [[[[[[...], [[...], '$31,536'], [[...], '$24,5... | 31536 |
| 2 | [[[[[[...], [[...], '$31,536'], [[...], '$24,5... | 24578 |
| 3 | [[[[[[...], [[...], '$31,536'], [[...], '$24,5... | 21461 |
| 4 | [[[[[[...], [[...], '$31,536'], [[...], '$24,5... | 11759 |
Click here if you need help locating the table
Below is the code to isolate the table, you will now need to loop through the rows and columns like in the previous lab
soup.find_all("tbody")[1]
If you want to use the read_html function the table is located at index 1
We are focusing on quarterly revenue in the lab.
Execute the following line to remove the comma and dollar sign from the Revenue column.
tesla_revenue["Revenue"] = tesla_revenue['Revenue'].str.replace(",","").str.replace("$", "")
Execute the following lines to remove an null or empty strings in the Revenue column.
# Remove empty strings
tesla_revenue = tesla_revenue[tesla_revenue["Revenue"] != ""]
# Drop null values
tesla_revenue.dropna(inplace=True)
Display the last 5 row of the tesla_revenue dataframe using the tail function. Take a screenshot of the results.
import requests
from bs4 import BeautifulSoup
import pandas as pd
url = "https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/revenue.htm"
html_data = requests.get(url).text
soup = BeautifulSoup(html_data, "html.parser")
tesla_revenue = pd.DataFrame(columns=["Date", "Revenue"])
for table in soup.find_all("table"):
if "Tesla Quarterly Revenue" in str(table):
for row in table.find("tbody").find_all("tr"):
col = row.find_all("td")
if len(col) != 0:
tesla_revenue = pd.concat(
[tesla_revenue,
pd.DataFrame({"Date":[col[0].text.strip()],
"Revenue":[col[1].text.strip()]})],
ignore_index=True
)
tesla_revenue["Revenue"] = tesla_revenue["Revenue"].str.replace(",", "").str.replace("$", "")
tesla_revenue.tail()
| Date | Revenue | |
|---|---|---|
| 49 | 2010-06-30 | 28 |
| 50 | 2010-03-31 | 21 |
| 51 | 2009-12-31 | |
| 52 | 2009-09-30 | 46 |
| 53 | 2009-06-30 | 27 |
import pandas as pd
import requests
from bs4 import BeautifulSoup
url = "https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/revenue.htm"
response = requests.get(url)
html_data = response.text
soup = BeautifulSoup(html_data, "html.parser")
import requests
url = "https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/revenue.html"
response = requests.get(url)
html_data = response.text
print(html_data[:500]) # print first 500 characters to see if it loaded correctly
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Resource>/cf-courses-data/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/revenue.html</Resource><RequestId>96bfaecf-6bba-40a6-9641-e21e08731079</RequestId><httpStatusCode>404</httpStatusCode></Error>
tesla_revenue.tail()
| Date | Revenue | |
|---|---|---|
| 5 | 2021-06-30 | 11958 |
| 6 | 2021-03-31 | 10389 |
| 7 | 2020-12-31 | 10744 |
| 8 | 2020-09-30 | 8771 |
| 9 | 2020-06-30 | 6036 |
!pip install lxml
Collecting lxml Downloading lxml-6.0.2-cp312-cp312-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl.metadata (3.6 kB) Downloading lxml-6.0.2-cp312-cp312-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl (5.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.3/5.3 MB 93.2 MB/s eta 0:00:00 Installing collected packages: lxml Successfully installed lxml-6.0.2
!pip install pandas
Requirement already satisfied: pandas in /opt/conda/lib/python3.12/site-packages (3.0.1) Requirement already satisfied: numpy>=1.26.0 in /opt/conda/lib/python3.12/site-packages (from pandas) (2.4.2) Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.12/site-packages (from pandas) (2.9.0.post0) Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.12/site-packages (from python-dateutil>=2.8.2->pandas) (1.17.0)
Question 3: Use yfinance to Extract Stock Data¶
Using the Ticker function enter the ticker symbol of the stock we want to extract data on to create a ticker object. The stock is GameStop and its ticker symbol is GME.
!pip install yfinance
Requirement already satisfied: yfinance in /opt/conda/lib/python3.12/site-packages (1.2.0) Requirement already satisfied: pandas>=1.3.0 in /opt/conda/lib/python3.12/site-packages (from yfinance) (3.0.1) Requirement already satisfied: numpy>=1.16.5 in /opt/conda/lib/python3.12/site-packages (from yfinance) (2.4.2) Requirement already satisfied: requests>=2.31 in /opt/conda/lib/python3.12/site-packages (from yfinance) (2.32.3) Requirement already satisfied: multitasking>=0.0.7 in /opt/conda/lib/python3.12/site-packages (from yfinance) (0.0.12) Requirement already satisfied: platformdirs>=2.0.0 in /opt/conda/lib/python3.12/site-packages (from yfinance) (4.3.6) Requirement already satisfied: pytz>=2022.5 in /opt/conda/lib/python3.12/site-packages (from yfinance) (2024.2) Requirement already satisfied: frozendict>=2.3.4 in /opt/conda/lib/python3.12/site-packages (from yfinance) (2.4.6) Requirement already satisfied: peewee>=3.16.2 in /opt/conda/lib/python3.12/site-packages (from yfinance) (3.19.0) Requirement already satisfied: beautifulsoup4>=4.11.1 in /opt/conda/lib/python3.12/site-packages (from yfinance) (4.12.3) Requirement already satisfied: curl_cffi<0.14,>=0.7 in /opt/conda/lib/python3.12/site-packages (from yfinance) (0.13.0) Requirement already satisfied: protobuf>=3.19.0 in /opt/conda/lib/python3.12/site-packages (from yfinance) (6.33.5) Requirement already satisfied: websockets>=13.0 in /opt/conda/lib/python3.12/site-packages (from yfinance) (16.0) Requirement already satisfied: soupsieve>1.2 in /opt/conda/lib/python3.12/site-packages (from beautifulsoup4>=4.11.1->yfinance) (2.5) Requirement already satisfied: cffi>=1.12.0 in /opt/conda/lib/python3.12/site-packages (from curl_cffi<0.14,>=0.7->yfinance) (1.17.1) Requirement already satisfied: certifi>=2024.2.2 in /opt/conda/lib/python3.12/site-packages (from curl_cffi<0.14,>=0.7->yfinance) (2024.12.14) Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.12/site-packages (from pandas>=1.3.0->yfinance) (2.9.0.post0) Requirement already satisfied: charset_normalizer<4,>=2 in /opt/conda/lib/python3.12/site-packages (from requests>=2.31->yfinance) (3.4.1) Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.12/site-packages (from requests>=2.31->yfinance) (3.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.12/site-packages (from requests>=2.31->yfinance) (2.3.0) Requirement already satisfied: pycparser in /opt/conda/lib/python3.12/site-packages (from cffi>=1.12.0->curl_cffi<0.14,>=0.7->yfinance) (2.22) Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.12/site-packages (from python-dateutil>=2.8.2->pandas>=1.3.0->yfinance) (1.17.0)
import yfinance as yf
# Create a ticker object for GameStop
gme = yf.Ticker("GME")
import yfinance as yf
gamestop = yf.Ticker("GME")
import yfinance as yf
Using the ticker object and the function history extract stock information and save it in a dataframe named gme_data. Set the period parameter to "max" so we get information for the maximum amount of time.
import yfinance as yf
# Create a ticker object for GameStop
gme = yf.Ticker("GME")
# Extract historical stock data for the maximum period
gme_data = gme.history(period="max")
# Reset the index so 'Date' becomes a column
gme_data.reset_index(inplace=True)
# Display the first 5 rows
gme_data.head()
| Date | Open | High | Low | Close | Volume | Dividends | Stock Splits | |
|---|---|---|---|---|---|---|---|---|
| 0 | 2002-02-13 00:00:00-05:00 | 1.620128 | 1.693350 | 1.603296 | 1.691666 | 76216000 | 0.0 | 0.0 |
| 1 | 2002-02-14 00:00:00-05:00 | 1.712707 | 1.716074 | 1.670626 | 1.683251 | 11021600 | 0.0 | 0.0 |
| 2 | 2002-02-15 00:00:00-05:00 | 1.683251 | 1.687459 | 1.658002 | 1.674834 | 8389600 | 0.0 | 0.0 |
| 3 | 2002-02-19 00:00:00-05:00 | 1.666418 | 1.666418 | 1.578047 | 1.607504 | 7410400 | 0.0 | 0.0 |
| 4 | 2002-02-20 00:00:00-05:00 | 1.615920 | 1.662210 | 1.603296 | 1.662210 | 6892800 | 0.0 | 0.0 |
import yfinance as yf
import pandas as pd
Reset the index using the reset_index(inplace=True) function on the gme_data DataFrame and display the first five rows of the gme_data dataframe using the head function. Take a screenshot of the results and code from the beginning of Question 3 to the results below.
# Reset the index so 'Date' becomes a column
gme_data.reset_index(inplace=True)
# Display the first five rows
gme_data.head()
| index | Date | Open | High | Low | Close | Volume | Dividends | Stock Splits | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 2002-02-13 00:00:00-05:00 | 1.620128 | 1.693350 | 1.603296 | 1.691666 | 76216000 | 0.0 | 0.0 |
| 1 | 1 | 2002-02-14 00:00:00-05:00 | 1.712707 | 1.716074 | 1.670626 | 1.683251 | 11021600 | 0.0 | 0.0 |
| 2 | 2 | 2002-02-15 00:00:00-05:00 | 1.683251 | 1.687459 | 1.658002 | 1.674834 | 8389600 | 0.0 | 0.0 |
| 3 | 3 | 2002-02-19 00:00:00-05:00 | 1.666418 | 1.666418 | 1.578047 | 1.607504 | 7410400 | 0.0 | 0.0 |
| 4 | 4 | 2002-02-20 00:00:00-05:00 | 1.615920 | 1.662210 | 1.603296 | 1.662210 | 6892800 | 0.0 | 0.0 |
import yfinance as yf
gme = yf.Ticker("GME")
gme_data = gme.history(period="max")
Question 4: Use Webscraping to Extract GME Revenue Data¶
Use the requests library to download the webpage https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/stock.html. Save the text of the response as a variable named html_data_2
import requests
# URL of the webpage
url = "https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/stock.html"
# Send GET request
response = requests.get(url)
# Save the text of the response
html_data_2 = response.text
# Optional: check the first 500 characters to confirm
print(html_data_2[:500])
<!DOCTYPE html> <!-- saved from url=(0105)https://web.archive.org/web/20200814131437/https://www.macrotrends.net/stocks/charts/GME/gamestop/revenue --> <html class=" js flexbox canvas canvastext webgl no-touch geolocation postmessage websqldatabase indexeddb hashchange history draganddrop websockets rgba hsla multiplebgs backgroundsize borderimage borderradius boxshadow textshadow opacity cssanimations csscolumns cssgradients cssreflections csstransforms csstransforms3d csstransitions fontface g
Parse the html data using beautiful_soup using parser i.e html5lib or html.parser.
from bs4 import BeautifulSoup
soup = BeautifulSoup(html_data_2, "html.parser")
Using BeautifulSoup or the read_html function extract the table with GameStop Revenue and store it into a dataframe named gme_revenue. The dataframe should have columns Date and Revenue. Make sure the comma and dollar sign is removed from the Revenue column.
Note: Use the method similar to what you did in question 2.
Click here if you need help locating the table
Below is the code to isolate the table, you will now need to loop through the rows and columns like in the previous lab
soup.find_all("tbody")[1]
If you want to use the read_html function the table is located at index 1
import pandas as pd
from bs4 import BeautifulSoup
import requests
url = "https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0220EN-SkillsNetwork/labs/project/stock.html"
html_data_2 = requests.get(url).text
soup = BeautifulSoup(html_data_2, "html.parser") # or "html5lib"
# Find all tables
tables = soup.find_all("table")
# Loop through tables to find one with "Revenue"
for table in tables:
if "Revenue" in table.text:
target_table = table
break
# Extract rows
rows = target_table.find_all("tr")
data = []
for row in rows[1:]: # skip header
cols = row.find_all("td")
if len(cols) == 2:
date = cols[0].text.strip()
revenue = cols[1].text.strip()
data.append([date, revenue])
# Create DataFrame
gme_revenue = pd.DataFrame(data, columns=["Date", "Revenue"])
# Clean Revenue column
gme_revenue["Revenue"] = gme_revenue["Revenue"].str.replace(r"[$,]", "", regex=True)
gme_revenue.head()
| Date | Revenue | |
|---|---|---|
| 0 | 2020 | 6466 |
| 1 | 2019 | 8285 |
| 2 | 2018 | 8547 |
| 3 | 2017 | 7965 |
| 4 | 2016 | 9364 |
Remove the comma and dollar sign, an null or empty strings from the Revenue column.
# Remove $ and commas
gme_revenue["Revenue"] = gme_revenue["Revenue"].str.replace(r"[$,]", "", regex=True)
# Remove rows where Revenue is empty or null
gme_revenue = gme_revenue[gme_revenue["Revenue"].notna()] # remove NaN
gme_revenue = gme_revenue[gme_revenue["Revenue"] != ""] # remove empty strings
# Optionally convert Revenue to numeric
gme_revenue["Revenue"] = pd.to_numeric(gme_revenue["Revenue"])
# Display first 5 rows
gme_revenue.head()
| Date | Revenue | |
|---|---|---|
| 0 | 2020 | 6466 |
| 1 | 2019 | 8285 |
| 2 | 2018 | 8547 |
| 3 | 2017 | 7965 |
| 4 | 2016 | 9364 |
Display the last five rows of the gme_revenue dataframe using the tail function. Take a screenshot of the results.
gme_revenue.tail()
| Date | Revenue | |
|---|---|---|
| 11 | 2009 | 8806 |
| 12 | 2008 | 7094 |
| 13 | 2007 | 5319 |
| 14 | 2006 | 3092 |
| 15 | 2005 | 1843 |
Question 5: Plot Tesla Stock Graph¶
Use the make_graph function to graph the Tesla Stock Data, also provide a title for the graph. Note the graph will only show data upto June 2021.
Hint
You just need to invoke the make_graph function with the required parameter to print the graphs.The structure to call the `make_graph` function is `make_graph(tesla_data, tesla_revenue, 'Tesla')`.
import plotly.graph_objects as go
from plotly.subplots import make_subplots
def make_graph(stock_data, revenue_data, stock):
fig = make_subplots(rows=2, cols=1, shared_xaxes=True,
subplot_titles=("Historical Share Price", "Historical Revenue"),
vertical_spacing = 0.1)
# Stock Data
stock_data_specific = stock_data[stock_data['Date'] <= '2021-06-30']
fig.add_trace(go.Scatter(x=stock_data_specific.Date,
y=stock_data_specific.Close.astype("float"),
name="Share Price"),
row=1, col=1)
# Revenue Data
revenue_data_specific = revenue_data[revenue_data['Date'] <= '2021-06-30']
fig.add_trace(go.Scatter(x=revenue_data_specific.Date,
y=revenue_data_specific.Revenue.astype("float"),
name="Revenue"),
row=2, col=1)
fig.update_layout(
showlegend=False,
height=700,
title=stock,
xaxis_title="Date",
yaxis_title="Price ($US)",
xaxis2_title="Date",
yaxis2_title="Revenue ($US Millions)"
)
fig.show()
make_graph(tesla_data, tesla_revenue, 'Tesla')
Question 6: Plot GameStop Stock Graph¶
Use the make_graph function to graph the GameStop Stock Data, also provide a title for the graph. The structure to call the make_graph function is make_graph(gme_data, gme_revenue, 'GameStop'). Note the graph will only show data upto June 2021.
Hint
You just need to invoke the make_graph function with the required parameter to print the graphs.The structure to call the `make_graph` function is `make_graph(gme_data, gme_revenue, 'GameStop')`
gme_data.reset_index(inplace=True) # Makes the index (date) into a column
print(gme_data.columns) # Should now include 'Date'
Index(['index', 'Date', 'Open', 'High', 'Low', 'Close', 'Volume', 'Dividends',
'Stock Splits'],
dtype='str')
make_graph(gme_data, gme_revenue, 'GameStop')
About the Authors:
Joseph Santarcangelo has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.
Azim Hirjani
Change Log¶
| Date (YYYY-MM-DD) | Version | Changed By | Change Description |
|---|---|---|---|
| 2022-02-28 | 1.2 | Lakshmi Holla | Changed the URL of GameStop |
| 2020-11-10 | 1.1 | Malika Singla | Deleted the Optional part |
| 2020-08-27 | 1.0 | Malika Singla | Added lab to GitLab |
Copyright © 2020 IBM Corporation. All rights reserved.